Towards the evaluation of the LarKC Reasoner Plug-ins
نویسنده
چکیده
In this paper, we present an initial framework of evaluation and benchmarking of reasoners deployed within the LarKC platform, a platform for massive distributed incomplete reasoning that will remove the scalability barriers of currently existing reasoning systems for the Semantic Web. We discuss the evaluation methods, measures, benchmarks, and performance targets for the plug-ins to be developed for approximate reasoning with interleaved reasoning and selection. In this paper, we propose a specification language of gold standards for the evaluation and benchmarking, and discuss how it can be used for the evaluation of reasoner plug-ins within the LarKC platform. keywords: Reasoning, Evaluation, Benchmarking, Web Scale Reasoning
منابع مشابه
The Large Knowledge Collider a Platform for Large Scale Integrated Reasoning and Web-search D4.7.1 Initial Evaluation and Revision of Plug-ins Deployed in Use-cases Quality Assessor: Mick Kerrigan(sti Innsbruck) Quality Controller: Frank Van Harmelen (vua) Executive Summary
for dissemination) The essence of the LarKC project is to go beyond notions of absolute correctness and completeness in reasoning. We are looking for retrieval methods that provide useful responses at a feasible cost of information acquisition and processing. Therefore, generic inference methods need to be extended to non-standard approaches. In consequence, traditional metrics such as complete...
متن کاملDeveloping an Incomplete Reasoner in Five Minutes: The Large Knowledge Collider in Action
The Large Knowledge Collider (LarKC) is a prominent development platform for the Semantic Web reasoning applications. Guided by the preliminary goal to facilitate the incomplete reasoning, LarKC has evolved in a unique platform, which can be used for the development of robust, flexible, and efficient semantic web applications, also leveraging the modern grid and cloud resources. As a reaction o...
متن کاملEvolved Framework for Measuring and Evaluating Heuristic Problem Solving
for dissemination) In this deliverable we refine the framework proposed in our previous Deliverable D1.4.1 [1] by providing a more formal framework for evalauting heuristic problem solving in LarKC. This evolved framework provides an abstract and formal definition of three of the measures proposed in the aforementioned document. More concretely, this document provides a formal definition of the...
متن کاملInitial Framework for Measuring and Evaluating Heuristic Problem Solving
for dissemination) One of the key aspects in the development of LarKC is how to evaluate the performance of the platform and its constituent components in order to guarantee that the execution of a pipeline will match the user’s needs and provide the desired solutions (answers) to the user’s queries. Therefore, in this deliverable, the first in a series three documents concerned with the defini...
متن کاملTowards A Dynamic Object-Oriented Design Metric Plug-in Framework
Problem Statement: The evolution of software is made difficult by the need to integrate new features with all previously implemented features in software applications. Approach: present study introduced a general-purpose, platform-independent object-oriented design metric plug-in framework called jmetric intended to help building scalable, extendable object-oriented design metric plug-ins. jmet...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2010